The term “optical flow” generally refers to the apparent motion of texture seen by an agent (such as an animal or a robot) as a result of relative motion between the agent and other objects in the environment. It is well known that animals, especially insects, use information from optical flow for depth perception and navigating through an environment without colliding into obstacles. An introduction to the concept of optical flow may be found in the book “The Ecological Approach to Visual Perception”, by J. Gibson, published by Lawrence Erlbaum Associates in 1986. Some examples of how insects utilize optical flow for navigation may be found in issue 199(1) of the Journal of Experimental Biology, edited by Wehner, Lehrer, and Harvey, and published in 1996 by The Company of Biologists Limited. Robotics and machine vision researchers have borrowed from these examples in biology to build machine vision systems that use optical flow for depth perception and obstacle avoidance in real environments. Examples of how optical flow may be used to perform some robotic tasks may be found in the paper “Biological Inspired Visual Sensing and Flight Control” by Barrows, Chahl, and Srinivasan, which appeared in the March 2003 issue of The Aeronautical Journal published by The Royal Aeronautical Society. Many other examples may be found in the academic literature.
The term “optical flow” is generally described in the academic literature as a vector field, with the domain of the vector field equal to the spherical visual field and the vectors representing the apparent velocity of visual texture in the visual field. In this document, the term “optical flow” will be used in a broader sense to include all types of visual motion that may be measured.
Machine vision systems capable of computing optical flow or visual motion in a compact package may be implemented with the use of specialized hardware. An imaging sensor is a device that is capable of sensing imagery based on light focused thereon. A machine visual sensing system or a vision sensor may be defined as an imaging sensory system having both image sensing and image processing functions, whether these functions are all performed primarily on a single chip or on a multiple chip system. In a “neuromorphic” approach, some or all computations may be performed with analog or mixed-signal circuits (i.e. mixed analog and digital) which exploit the physical dynamics inherent in VLSI (very large scale integration) circuitry and may additionally mimic biological structures. One source that provides such “neuromorphic” approaches is a book entitled “Analog VLSI and Neural Systems” by C. Mead, published by Addison Wesley in 1989. A related approach is to use “vision chips”, which are defined herein to be integrated circuits having both image acquisition circuitry and image processing circuitry in the same device, including within the same monolithic die. One book that provides methods of implementing vision chips is “Towards the Visual Microprocessor” edited by T. Roska and A. Rodríguez-Vázquez, and published by Wiley in 2001. Another book that provides methods of implementing vision chips is “Vision Chips” by A. Moini and published by Kluwer Academic Publishing in 1999. When either of the above mentioned approaches is properly executed, it is possible to implement a machine vision system capable of performing a given set of tasks in a package substantially smaller than that when utilizing a conventional CMOS (complementary metal-oxide-semiconductor) or CCD (charge coupled device) imager connected to a high-performance processor.
In U.S. Pat. No. 6,020,953, a motion sensing apparatus and method, referred herein as the “competitive feature tracker” (CFT) algorithm, are disclosed. The approach described therein can be implemented using a vision chip. One instance of the aforementioned apparatus or algorithm may be referred to as an “elementary motion detector” or “EMD” that produces an optical flow measurement computed based on the motion of a specific feature across the visual field. Although a device implementing this apparatus or algorithm may be able to measure optical flow when exposed to real-world textures and, hence, able to measure optical flow even when the texture contrast is significantly lower than what is normally found in nature, the device may make occasional erroneous measurements especially as the texture contrast becomes even lower.
To address this issue, a method of fusing EMD outputs in order to increase the reliability of the optical flow measurement was developed, which is disclosed in U.S. Pat. No. 6,384,905. An optical flow sensor as described therein utilizes multiple arrays of feature detectors, with each array configured to track a different feature across the visual field. This sensor implements multiple EMDs that monitor the same portion of the visual field, with each EMD monitoring the motion with respect to a different feature. The output of each individual EMD may be referred to as a “velocity report” since it is a single report of the measured optical flow velocity. The sensor then combines the velocity reports produced by different EMDs in such a manner that individual erroneous velocity reports are filtered out. The sensor then produces a single optical flow measurement that is statistically more reliable.
Further details on the implementation of the teachings of the two aforementioned patents may be found in the following papers: “Feature Tracking Linear Optic Flow Sensor Chip” by Miller and Barrows and included in the proceedings of the IEEE 1999 International Symposium on Circuits and Systems (ISCAS '99); “Fusing Neuromorphic Motion Detector Outputs for Robust Optical Flow Measurement” by Barrows, Miller, and Krantz and included in the proceedings of the IEEE 1999 International Joint Conference on Neural Networks (ISCAS '99); and “Mixed-Mode VLSI Optical Flow Sensors for In-Flight Control of a Micro Air Vehicle” by Barrows and Neely and included in SPIE Volume 4109, published by SPIE in 2000. Further details may also be found in the Ph.D. dissertation entitled “Mixed-Mode VLSI Optic Flow Sensors for Micro Air Vehicles” written by Barrows in 1999 at the University of Maryland at College Park. Additional details may be found in the book chapter “Optical Flow Sensors for UAV Navigation” by Barrows, Neely, and Miller, which was published as part of the book entitled “Fixed and Flapping Wing Aerodynamics for Micro Air Vehicle Applications” and published by AIAA in 2001.
Although the sensors described in the two aforementioned patents are practical, there remains an issue related to the aspect of the CFT algorithm disclosed in the aforementioned patents. In practice, additional steps are necessary to perform arbitration (or conditioning) on the feature location signals to handle cases when more than one feature location signal is concurrently high. Arbitration may be performed based on a combination of low-pass filters and arbitration logic, which require extra instructions and/or additional circuitry to implement. However, without such arbitration, the sensor's performance degrades. To minimize the amount of circuitry on a chip or device, a method to achieve reliable performance without arbitration is needed.